Manipulating Boxes Using A Zoned Gripper

ABSTRACT

A method of manipulating boxes includes receiving a minimum box size for a plurality of boxes varying in size located in a walled container. The method also includes dividing a grip area of a gripper into a plurality of zones. The method further includes locating a set of candidate boxes based on an image from a visual sensor. For each zone, the method additionally includes, determining an overlap of a respective zone with one or more neighboring boxes to the set of candidate boxes. The method also includes determining a grasp pose for a target candidate box that avoids one or more walls of the walled container. The method further includes executing the grasp pose to lift the target candidate box by the gripper where the gripper activates each zone of the plurality of zones that does not overlap a respective neighboring box to the target candidate box.

CROSS REFERENCE TO RELATED APPLICATIONS

This U.S. patent application is a continuation of, and claims priorityunder 35 U.S.C. § 120 from, U.S. patent application Ser. No. 16/538,114,filed on Aug. 12, 2019, which claims priority under 35 U.S.C. § 119(e)to U.S. Provisional Application 62/718,060, filed on Aug. 13, 2018. Thedisclosures of these prior applications are considered part of thedisclosure of this application and are hereby incorporated by referencein their entireties.

TECHNICAL FIELD

The present disclosure is directed toward manipulating boxes using azoned gripper.

BACKGROUND

Box-like objects represent a large percentage of objects that need to bepicked (i.e., removed from a pallet or holding container) in industrial,manufacturing, logistics, and commercial environments. Typically,box-like objects are characterized by at least one substantially planarpicking surface. Conventionally, during robotic picking, a picking robothandles known sizes, numbers, and types of boxes arranged in a uniformmanner on a structured pallet. Using mechanical fixtures, some currentsystems pre-position a pallet of boxes so that a robot can pick themfrom known pre-programmed locations. Any deviation from this knownstructure, either in the size of the box, the number of boxes, or thelocation of boxes results in failure of the system. Unfortunately,computer-vision-based systems often rely on the boxes having clean edgesat their boundaries and cannot accurately determine size and/or positionof boxes that have advertising, printed characters, printed logos,pictures, color, or any other texture on them. Such boxes have visualedges on their faces (i.e., edges that do not correspond to an actualphysical boundary of the box). Because current computer-vision-basedsystems cannot distinguish the physical edges between two differentboxes from other visual edges on the faces of boxes, these systems tendto misjudge the size and position of the box(es). Problematically,picking and moving the box where the system has misjudged its size andlocation may either cause the box to slip from the grasp of the robot ormay cause the robot to pick two or more boxes where it should havepicked only one.

SUMMARY

One aspect of the disclosure provides a method of manipulating boxesusing a zoned griper. The method includes receiving, at a controlsystem, a minimum box size for a plurality of boxes varying in size.Here, the plurality of boxes are located in a walled container. Therobot includes a gripper with a plurality of vacuum suction cups. Themethod also includes dividing, by the control system, a grip area of thegripper into a plurality of zones based on the minimum box size. Themethod further includes locating, by the control system, a set ofcandidate boxes of the plurality of boxes based on an image from avisual sensor. For each zone of the plurality of zones, the methodadditionally includes, determining, by the control system, an overlap ofa respective zone with one or more neighboring boxes to the set ofcandidate boxes where the neighboring boxes identified by the image fromthe visual sensor. The method also includes determining, by the controlsystem, a grasp pose for a target candidate box of the set of candidateboxes that avoids one or more walls of the walled container. The methodfurther includes executing, by the control system, the grasp pose tolift the target candidate box by the gripper where the gripper activateseach zone of the plurality of zones that does not overlap a respectiveneighboring box to the target candidate box.

Implementations of the disclosure may include one or more of thefollowing optional features. In some implementations, the method alsoincludes determining, by the control system, that the grasp poseincludes minimum coverage of the target candidate box where the minimumcoverage corresponds to an area providing suction force sufficient tolift the target candidate box. In some examples, the method additionallyincludes determining, by the control system, that a part-presence sensoroverlaps with the target candidate box by a sufficient margin where thesufficient margin triggers the part-presence sensor to communicate apresence of the target candidate box. Determining the grasp pose fortarget candidate box may include offsetting the grasp pose based on alocation of the one or more walls of the walled container. In someconfigurations, the method also includes determining, by the controlsystem, that the gripper has lifted the target candidate box a thresholdheight and activating, by the control system, all zones of the gripper.In some examples, the method further includes identifying, by thecontrol system, a feature of the walled container to avoid duringremoval of the target candidate box from the walled container anddetermining, by the control system, a motion path for removal of thetarget candidate box from the walled container where the motion pathavoids the identified feature of the walled container.

Optionally, the plurality of zones may cascade across the plurality ofvacuum suction cups of the gripper. The plurality of boxes varying insize may correspond to rectilinear objects. In some example, theplurality of boxes varying in size corresponds to the plurality of boxesvarying in height. The grasp pose may correspond to a position of thegripper defined by a set of vacuum suction cups of the plurality ofvacuum suction cups overlapping a top surface of a target candidate box.

Another aspect of the disclosure provides a robot that manipulates boxesusing a zoned griper. The robot includes a visual sensor, a gripper, anda control system. The gripped includes a plurality of vacuum suctioncups and the control system is configured to perform operations. Theoperations include receiving a minimum box size for a plurality of boxesvarying in size, the plurality of boxes located in a walled container.The operations also include dividing a grip area of the gripper into aplurality of zones based on the minimum box size. The operations furtherinclude locating a set of candidate boxes of the plurality of boxesbased on an image from the visual sensor. For each zone of the pluralityof zones, the operations additionally include determining an overlap ofa respective zone with one or more neighboring boxes to the set ofcandidate boxes, the neighboring boxes identified by the image from thevisual sensor. The operations also include determining a grasp pose fora target candidate box of the set of candidate boxes that avoids one ormore walls of the walled container. The operations further includeexecuting the grasp pose to lift the target candidate box by the gripperto the control system, the grasp pose actuating each zone of theplurality of zones that does not overlap a respective neighboring box tothe target candidate box.

This aspect may include one or more of the following optional features.In some implementations, the operations also include determining thatthe grasp pose includes minimum coverage of the target candidate boxwhere the minimum coverage corresponds to an area providing suctionforce sufficient to lift the target candidate box. In some examples, theoperations additionally include determining that a part-presence sensoroverlaps with the target candidate box by a sufficient margin where thesufficient margin triggers the part-presence sensor to communicate apresence of the target candidate box. Determining the grasp pose fortarget candidate box may include offsetting the grasp pose based on alocation of the one or more walls of the walled container. In someconfigurations, the operations also include determining that the gripperhas lifted the target candidate box a threshold height and activating,by the control system, all zones of the gripper. In some examples, theoperations further include identifying a feature of the walled containerto avoid during removal of the target candidate box from the walledcontainer and determining a motion path for removal of the targetcandidate box from the walled container where the motion path avoids theidentified feature of the walled container.

The details of one or more implementations of the disclosure are setforth in the accompanying drawings and the description below. Otheraspects, features, and advantages may be apparent from the descriptionand drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic view of an example robot with a zoned gripper inan environment with boxes.

FIG. 2A is a schematic view of an example conventional arrangement for agripper.

FIGS. 2B-2D are schematic views of example zones for the zoned gripperof FIG. 1.

FIGS. 3A-3J are schematic views of example grasp poses for a targetcandidate box.

FIGS. 4A-4D are schematic views of example zone gripper actuations withrespect to a target candidate box.

FIGS. 5A and 5B are cross-sectional views of a container with examplemovement paths to remove a box from within the container.

FIG. 6 is a schematic view of an example picking environment with aconveyor.

FIG. 7 is a flow diagram of an example arrangement of operations for amethod of using a zoned gripper to grasp and move a target candidatebox.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

Implementations herein are directed toward robotic picking softwaresolutions and zoned grippers that can reliably pick individual boxes ofvarying sizes, shapes, and/or orientations from an enclosed pallet orcontainer with walls on one or more sides. Although U.S. PatentPublication No. 2017/0246744 describes systems that can pick frommixed-stock keeping unit (SKU) pallets, (e.g., pallets with boxes ofdifferent sizes), an additional set of issues arises when trying to pickfrom mixed-SKU containers (i.e., containers with at least one wall thatcontain boxes of different sizes). Since portions of the robotic pickingprocess utilize aspects of picking mixed-SKUs, U.S. Patent PublicationNo. 2017/0246744 is hereby incorporated by reference in its entirety.

Robotic picking of boxes in industrial and/or logistics environments isusually performed using fixed-size single-zone grippers. Here, when theboxes are of different fixed sizes, the position of the vacuum grippercan be adjusted to overlap with only the box to be picked. Generally,the weight and size of boxes being picked and the speed at which theyneed to be moved determines the size of the vacuum gripper. For example,U.S. Patent Publication No. 2017/0246744 includes examples of such aprocess for picking mixed-SKU boxes from open pallets.

Picking individual boxes from a container or structure with wallsrequires a picking robot to avoid collisions with the walls and pick asingle box with minimal disturbance to the other contents of thecontainer. In this scenario, when picking boxes of different sizes, thegripper cannot often be placed to overlap a single box without collidingwith a wall of the container.

Referring to FIG. 1, an environment 10 includes a pallet 20 supporting acontainer 30 where the container 30 includes a plurality of mixed-SKUboxes 22, 22 a-n. As shown by the dotted hidden lines, the sizes (e.g.,dimensions) of the boxes 22 may vary. Here, the container 30 includesfour walls 30 w ₁₋₄ and a lip 32 around the top of the walls 30 w ₁₋₄opposite the pallet 20. The lip 32 is shown to extend towards a centerof the stack of boxes 22. The stack of boxes 22 may have one or morelevels (e.g., shown as three levels) where, on a top level, top faces ofthe boxes 22 face a part of a robot 100 (i.e., the gripper 200 of therobot 100). Although the container 30 is shown with multiple walls 30 w,the container 30 may be any vessel (e.g., bin, bulk bin, skid box,octabin, gaylord box, cage, crate, truck bed, trailer, shippingcontainer, compartment, etc.) with at least one wall that the robot 100should avoid when picking boxes 22 in the environment 10.

The environment 10 also includes a robot 100 with a control system 110,a visual sensor 120, and an arm 130 with a gripper 200. While the robot100 is depicted as a stationary robot 100, the robot 100 may include amobile robot without departing from the scope of the present disclosure.The control system 110 is configured to control the robot 100 by, forexample, operation of the gripper 200 to manipulate boxes 22 in theenvironment. The control system 110 is configured to communicate with acomputing device 40. The computing device 40 may be part of the controlsystem 110 or external to the control system 110 (e.g., remote from thecontrol system 110) as shown in FIG. 1. The computing device 40 mayenable automated control of the robot 100 via its communication with thecontrol system 110. The computing device 40 may also provide one or moreuser interfaces for an operator to interact with or to operate/controlthe robot 100. The computing device 40 may be any of a variety ofcomputing devices 40 and may be connected directly to the control system110, or may communicate with the control system 110 via interveningdevices and networks.

It should also be noted that, despite any references to particularcomputing paradigms and software tools herein, the computer programinstructions on which various implementations are based may correspondto any of a wide variety of programming languages, software tools anddata formats, may be stored in any type of non-transitorycomputer-readable storage media or memory device(s), and may be executedaccording to a variety of computing models including, for example, aclient/server model, a peer-to-peer model, on a stand-alone computingdevice, or according to a distributed computing model in which variousfunctionalities may be effected or employed at different locations.Suitable alternatives known to those of skill in the art may beemployed.

The visual sensor 120 is disposed at a location to observe the boxes 22to be picked by the robot 100. For instance, as shown in FIG. 1, thevisual sensor 120 is mounted on or near the gripper 200 to observe theboxes 22 from above (e.g., with a top-down view relative to the boxes22). Although the visual sensor 120 is mounted to the gripper 200 in theexample shown, the visual sensor 120 could be applied in multiple sensorplacement configurations, including cases in which the sensor(s) 120 aremounted on a fixed platform, or in which the sensor(s) 120 are mountedon the gripper 200 or other part of the robot arm 130, or in which thesensor(s) 120 are mounted on a moving platform (mobile robot) on whichthe robot arm 130 is also mounted. Other variations will be apparent tothose of skill in the art.

Referring to FIG. 2A, zoned vacuum grippers may include multiple suctioncup zones Z that can be individually controlled to allow a gripper toturn each zone Z on or off when desired independent of the other zonesZ. For instance, FIG. 2A illustrates six zones Z, Z₁₋₆ (shown indifferent shades of gray) where each zone Z includes nine suction cups.To pick up boxes 22 from a pallet 20, zoned grippers have been usedmostly to pick from single-SKU pallets where the sizes of the boxes arepreviously known. By knowing the sizes of the boxes beforehand, thezones Z may be explicitly designed to conform to the sizes of the boxes22 on the pallet 20. When transporting boxes 22 from a single-SKU pallet20, all zones Z of a zoned gripper 200 may be activated (e.g., turnedon) to permit the picking multiple boxes that overlap with the gripper200. Individual zones Z may then deactivate (e.g., switch off) insequence to drop individual boxes 22 off on a conveyor 50 (FIG. 6).

Unfortunately, for mixed-SKU pallets 20, there is typically little or noprior information available for the mix of different sizes and/orarrangements of boxes 22 on the mixed-SKU pallets 20. Furthermore, bytheir very nature, mixed-SKU pallets 20 have mixed layers of boxes 22and may contain boxes 22 of different heights in the same layer.Positioning a zoned gripper 200 in this scenario may require: (a)choosing/selecting the right/correct box 22 to pick up among multipleboxes in the same layer; (b) correctly selecting the zones Z that needto be switched on for picking the selected box 22; and (c) planningand/or executing a path 500 that permits removal of the selected box 22from the container 30 while avoiding contact/interference with the walls30 w and/or other boxes 22.

For robustness, grippers 200 often include off-the-shelf part-presencesensors 220 (e.g. optical sensors) that are built into the gripper 200to detect the presence or absence of a box 22 after picking the box 22.This allows for independent confirmation of the success or failure of agrasp by the robot 100. For mixed-SKU box picking, the part-presencesensor 220 needs to be properly positioned relative to the box 22 beingpicked so that the part-presence sensor 220 will react appropriately tothe presence or absence of the box 22.

Implementations herein are directed toward using a robot 100 to pickindividual boxes 22 from a mixed-SKU collection of boxes 22 in acontainer 30, cage, or any kind of structure with at least one wall 30w, feature, or other component that may potentially obstruct the motionof the robot 100 while picking the boxes 22.

In some implementations, the only prior information available to thesystem (e.g., the control system 110) is the minimum and maximum sizesof the boxes 22 to be picked. The zoned gripper 200 may be configured asa cascaded set of zones Z (e.g., based on the minimum and maximumsizes). For instance, FIGS. 2B-2D depict the zoned gripper 200 includingthree zones Z, Z₁₋₃ to form a cascaded set of zones Z. Moreparticularly, FIG. 2B illustrates the first zone Z, Z₁ of the cascadedset of zones Z including three rows and three columns of suction cups210 switched on in the top left corner of the gripper 200 relative tothe view of FIG. 2B. FIG. 2C shows that a second zone Z, Z₂ for thegripper 200 includes different dimensions than the first zone Z₁ byincluding an L-shaped configuration that includes switching on the nextrow and column of suction cups 210 of the gripper 200. FIG. 2D issimilar to FIG. 2C such that a third zone Z, Z₃ does not includeswitching on any suction cups 210 from the prior zones Z and includes alarger L-shaped configuration than the second zone Z₂ by switching onthe next row and column of suction cups 210 of the gripper 200.

In some examples, the overall size of the gripper 200 is determined bythe size of the largest box 22 to be picked. For example, the gripper200 in FIGS. 2B-2D includes a five by five (5×5) arrangement of suctioncups because the largest box 22 has a size that is less than or equal tothe five by five arrangement of suction cups 210. The smallest zone Zfor which the gripper 200 may be configured may be based on the size ofthe smallest box 22 to be picked.

In additional implementations, the control system 110 uses data obtainedfrom the visual sensor 120 mounted above the container 30 (e.g.,overhead and/or on the robot 100 itself) to locate a set of candidateboxes 24, 24 _(set) (e.g., shown in FIG. 1) on the top of the stack ofboxes 22. The boxes 22 may vary in size, have different visualappearances (e.g., colors, textures, designs, etc.), and could havedifferent heights in each layer as well. In addition, the boxes 22and/or their respective surfaces may be at varied orientations inthree-dimensional space (e.g., not necessarily limited to rectilinearobjects placed flat inside the container).

Referring to FIGS. 3A-3J, the control system 110 and/or computing device40 may compute a set of grasp poses P_(G) for each candidate box 24. Agrasp pose P_(G) is a position of the gripper 200 where a subset or allof the suction cups 210 on the gripper 200 overlap (partly or fully) thetop surface of the object being picked. Multiple grasp poses P_(G) maybe computed for each box 24 starting, for example, with full overlapwith one corner of the gripper 200 placed over each of the corners ofthe box 24 to be picked, but also including partial overlap where onlyparts of the box 24 being picked are covered by the suction cups 210.For example, FIGS. 3A and 3B illustrate a single grasp pose P_(G) for atarget candidate box 24, 24T where the grasp pose P_(G) fully overlaps asmaller target candidate box 24, 24T. Here, in FIG. 3A, the grasp poseP_(G) aligns with a top and left edge of the target candidate box 24,24T while in FIG. 3B, the grasp pose P_(G) aligns with the bottom andright edge of the target candidate box 24, 24T. The part-presence sensor220 is shown throughout FIGS. 3A-3J as a circular mark (e.g., similar toa fiducial registration target). In some implementations, such as inFIG. 3C, the control system 110 and/or computing device 40, generate anoffset grasp pose P_(Goff) (e.g., based off a grasp pose P_(G)) as analternative grasp pose P_(G) to lift the target candidate box 24, 24T.For instance, FIG. 3C depicts two offset grasp poses P_(Goff,1-2). InFIGS. 3C and 3D, neither grasp pose P_(G) fully overlaps the targetcandidate box 24, 24T; therefore, these grasp poses P_(G) are partiallyoverlapping grasp poses P_(G).

In some examples, when the target candidate box 24, 24T is smaller thanthe gripper 200 and inside the container 30 as shown in FIG. 3E, thecontrol system 110 and/or computing device 40 is configured to generatea plurality of grasp poses P_(G) (e.g., including offset grasp posesP_(Goff)). For instance, generating the plurality of grasp poses P_(G)may include generating an offset grasp pose P_(Goff) for each candidategrasp pose P_(G) (e.g., FIG. 3E shows three grasp poses P_(Goff1-3) withthree candidate grasp pose P_(G1-3)). By having a plurality of graspposes P_(G) and/or offset grasp poses P_(Goff), the gripper 200 may haveoptions to avoid a collision or disturbance with the container 30 duringpicking the target candidate box 24T.

The computation of a viable grasp pose P_(G) also accounts for thepresence of the walls 30 w of the container 30. When the gripper 200 isin a viable grasp pose P_(G), it is preferable that it not collide withthe walls 30 w (or other boxes 22). Grasp poses P_(G) may also be offsetin orientation from the orientation of the box 22 and the orientation ofthe container 30 (e.g., as shown in FIG. 3E). In contrast to FIG. 3E,FIG. 3F shows an example where the gripper 200 is smaller than thetarget candidate box 24T (i.e., the target candidate box 24, 24T islarger than the gripper 200). Here, the control system 110 and/orcomputing device 40 is also configured to generate a plurality or set ofgrasp poses P_(G) (e.g., shown as four candidate grasp poses P_(G1-4))that are sufficient to lift the target candidate box 24, 24T. FIG. 3Falso depicts the lip 32 of the container 30 to illustrate a potentialobstacle for the gripper 200 when maneuvering boxes 22 inside thecontainer 30.

Similarly, FIGS. 3G-3J illustrates examples where the gripper 200 issmaller than the target candidate box 24T. These examples depict thatthe location of the part-presence sensor 220 may change depending on,for example, the grasp pose P_(G) or the size of the target candidatebox 24, 24T. Even with changing the location of the part-presence sensor220 for each grasp pose P_(G), the computing device 40 and/or controlsystem 110 is configured to check whether the part-presence sensor 220overlaps with the box 24T by a sufficient margin. Here, overlap with thebox 24T by the sufficient margin allows triggering the part-presencesensor 220 when the box is close to the sensor 220.

Referring back to FIGS. 3A-3J, regardless of the resultant number ofcandidate grasp poses P_(G), every grasp pose P_(G) may be restricted tohave a minimum coverage of the box 24, 24T being picked to ensure thatthere is sufficient suction force to pick the box 22. Additionally oralternatively, for each grasp pose P_(G), the control system 110 and/orcomputing device 40 may compute the overlap of various gripper zones Zwith the box 24, 24T to be picked. In some implementations, systems forthe robot 100 (the control system 110 and/or computing device 40)compute the overlap of each gripper zone Z with neighboring boxes 26that are not candidates for picking. For instance, for each grasp poseP_(G), the systems use a list of boxes 22 provided by the visioncomponent 120 to compute the overlap of each gripper zone Z withneighboring boxes 26 that are not candidates for picking.

When there is no overlap between the gripper zones Z and neighboringboxes 26 that are not candidates for picking, all zones Z in the vacuumgripper 200 may activate (e.g., switch on) during the picking processfor a particular candidate box 24, 24T. For instance, FIGS. 4A-4C depicta single target candidate box 24, 24T that activates the cascading setof zones Z, Z₁₋₃ of FIGS. 2B-2D. For example, the target candidate box24, 24T in FIG. 4A includes a size requiring activation of only a firstzone Z₁. In FIG. 4B, the target candidate box 24, 24T includes a largersize requiring activation of both the first zone Z₁ and the second zoneZ₂ to lift the target candidate box 24, 24T. In FIG. 4C, the targetcandidate box 24, 24T is larger than the sizes of FIGS. 4A and 4B,thereby requiring activation of all zones Z₁₋₃ to lift the targetcandidate box 24, 24T. This tracking of overlap with neighboring boxes26 that are not candidates for picking advantageously provides thegripper with maximum picking coverage for each individual box 24 in amixed-SKU collection.

When there is overlap between the zones Z and neighboring boxes 26 thatare not candidates for picking, any zone Z that overlaps a neighboringbox 26 is not initially activated when picking an individual box 22. Thesmallest zone Z overlapping the target candidate box 24T is alwaysactivated (e.g., switched on) upon initially grasping the targetcandidate box 24T. Once the individual target candidate box 24T has beenpicked up and moved by a small distance (e.g., some threshold distance),additional (up to all) zones Z may be switched on, thereby ensuringmaximum coverage for grasping the box 24T. Referring to FIG. 4D, in someimplementations, when a neighboring box 26 overlaps one or more zones Zof the gripper 200 (e.g., the second zone Z₂ and the third zone Z₃)needed for providing maximum picking coverage based on a size of atarget candidate box 24T, the control system 110 and/or computing device40 initially only activates the first zone Z₁ to grasp the targetcandidate box 24T while leaving the first and second zones Z₂, Z₃deactivated. Here, once the gripper 200 has lifted the target candidatebox 24T to a particular height with only the first zone Z₁ activated,the control system 110 and/or computing device 40 may then activate allzones Z (e.g., maintain activation of the first zone Z₁ and switch onthe second and third zones Z₂, Z₃. The particular height may include athreshold height that prevents grasping of the one or more neighboringboxes 26 upon activation of the overlapping zones Z.

When a grasping planning phase is complete (i.e., with each grasp poseP_(G) computed and a selection of zones Z to be activated for aparticular box 24T determined), the system (e.g., the control system 110or the computing device 40) then proceeds to a path planning stage.FIGS. 5A and 5B show an example robot environment 10 depicting across-sectional view of a container 30 holding boxes 24, 26 of varyingsize. In the path planning stage, the system computes a path 500 toremove the object (i.e., a box 22) from the container 30 for transportto a target location. For instance, the robot 100 may remove a targetcandidate box 24T from a pallet 30 and place the target candidate box24T onto a conveyor 50 (FIG. 6). In some examples, when computing thepath 500, the system accounts for the size of the target candidate box24T being picked (according to some implementations postulating itsheight to be a fixed maximum height).

The first part of the computed path 500 is a straight line or set ofstraight line motions of the box 24T where it is moved away from theother boxes 22 in the container 30 and/or from the container walls 30 w.This motion is computed by taking into account the nature of thecontainer 30, e.g., some containers have a “lip” 32 or protrusion of thewalls 30 w. To reduce the likelihood of collision with features of thecontainer 30 while moving the target candidate box 24T, the systemcomputes the motion of the target candidate box 24T to avoid a collisionwith the features of the container 30. For example, in the case of a lip32 (e.g., as shown in FIGS. 5A and 5B), the computed motion of a pickedbox 24T might be a move inward towards a center of the container 30 byat least a distance greater than (or equal to) a width of the lip 32while also moving up towards the top of the container 30. The amount ofupward motion may be adjusted based on the position of the box 24T inthe container 30 as shown comparatively between FIGS. 5A and 5B. Forinstance, FIG. 5A shows that boxes 24T closer to the lip 32 are movedmostly sideways to clear the lip 32 (e.g., as shown in FIG. 5A). On theother hand, FIG. 5B shows that boxes 22, 24T positioned closer to thebottom of the container 30 are moved upward and inward towards thetop-center of the container 30 (e.g., as shown in FIG. 5B). Here, therobot 100 accounts for the height needed to travel with the box 24T toremove the box 24T from the container 30 when avoiding a feature of thecontainer 30, such as a lip 32. In this example, the robot 100 may movethe box 24T along two axis simultaneously and potentially minimizesudden movements when avoiding features of the container 30. In someimplementations, the movement path 500 for moving each box 24T includesfirst moving (e.g., straight up) the box 24T to clear other boxes 22(e.g., neighboring boxes 26) before moving inward to the top-center ofthe container 30 to clear the lip 32 and/or walls 30 w. Additionally oralternatively, once the robot 100 moves boxes 24T out of the container30, the robot 100 can move the boxes 24T to the target location (e.g., aconveyor 50).

Referring to FIG. 6, a conveyor sensor 300 (e.g., a distance/positionsensor such as a laser distance sensor) is mounted underneath theconveyor 50 and configured to measure a height H of a bottom surface ofa box 24T relative to the conveyor sensor 300 as a gripper 200 moves thebox 24T to the conveyor 50. Here, the height H refers to a distancebetween the bottom surface of the box 24T and the support surface of theconveyor 50 (e.g., conveyor belt). This data conveying the height H issynchronized with information about the position of the robot 100 atthat moment. For instance, the control system 110 is aware of a positionand/or a location of a top surface of the box 24T. More specifically,the robot 100 identifies a location on the box 24T to lift the box 24Taccordingly to the grasp pose P_(G) (e.g., by using the visual sensor120). In some configurations, with the height H from the conveyor sensor300, the location of the gripper 200 to engage the top surface of thebox 24T, and the control coordinates for movement of the box 24T (e.g.,along the path 500 and/or to the conveyor 50), the robot 100 (e.g., atthe control system 110 or the computing device 40) determines the heighth of the box 24T_(h). In other words, the system approximately infersthe height 24T_(h) of the box 24T based on its relationship to theconveyor sensor 300 and other control data (e.g., kinematics and/ordynamics) about the robot 100. The robot 100 may use the height H and/orthe box height 24T_(h) to minimize a drop height above the conveyor 50where the robot 100 deactivates one or more zones Z of the gripper 200to release the box 24T; therefore, generating a soft landing for the box24T. By minimizing the drop height, the robot 100 may prevent damage tothe box 24T and/or items contained within the box 24T. Here, the dropheight may be the equivalent of the height H, which may or may notinclude an offset distance accounting for the mounting position of theconveyor sensor 300 with respect to a surface of the conveyor 50. Insome examples, the control system 110 and/or computing device 40modifies the path 500 for placing the box 24T on the conveyor 50 so thateach box 22 is released approximately the same distance from theconveyor 50. Although FIG. 6 illustrates the soft landing technique withrespect to a conveyor 50, this concept may be translated to anyplacement surface where the robot 100 intends to release the box 24Tfrom its grasp. In some implementations, the conveyor sensor 300 includeultrasonic sensors, 3D camera, stereo vision or any other distancemeasuring device.

FIG. 7 is a flowchart of an example arrangement of operations for amethod 700 of using a zoned gripper 200 to grasp and move a targetcandidate box 24T. At operation 702, the method 700 receives, at system(e.g., the control system 110 or the computing device 40) of a robot100, a minimum box size for a plurality of boxes 22 varying in sizewhere the plurality of boxes 22 are located in a walled container 30.Here, the robot 100 includes a gripper 200 with a plurality of vacuumsuction cups 210. At operation 704, the method 700 divides a grip areaof the gripper 200 into a plurality of zones Z based on the minimum boxsize. At operation 706, the method 700 locates a set of candidate boxes24 of the plurality of boxes 22 based on an image from a visual sensor120. For each zone Z of the plurality of zones Z, at operation 708, themethod 700 determines an overlap with a respective zone Z with one ormore neighboring boxes 26 to the set of candidate boxes 24. Here, theneighboring boxes 26 are identified by the image from the visual sensor120. At operation 710, the method 700 determines a grasp pose P_(G) fora target candidate box 24T of the set of candidate boxes 24 that avoidsone or more walls 30 w of the walled container 30. At operation 712, themethod 700 executes the grasp pose P_(G) to lift the target candidatebox 24T by the gripper 200. Here, the gripper 200 activates each zone Zof the plurality of zones Z that does not overlap a respectiveneighboring box 26 to the target candidate box 24T.

It should be noted that implementations are contemplated in which therobot 100 may be in any of a variety of orientations relative to thecontainer and may even, in some instances, be mobile. Different robot100 types may also be employed (e.g., serial arm robots, parallel or“delta” robots, etc.).

It will be understood by those skilled in the art that changes in theform and details of the implementations described herein may be madewithout departing from the scope of this disclosure. In addition,although various advantages, aspects, and objects have been describedwith reference to various implementations, the scope of this disclosureshould not be limited by reference to such advantages, aspects, andobjects.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. Accordingly, otherimplementations are within the scope of the following claims.

What is claimed is:
 1. A computer-implemented method when executed bydata processing hardware of a robot causes the data processing hardwareto perform operations comprising: receiving a minimum object size for aplurality of objects varying in size, each object of the plurality ofobjects having at least one planar surface, the robot comprising agripper with a plurality of vacuum suction cups; dividing a grip area ofthe gripper into a plurality of zones based on the minimum object size;locating, using sensor data, a target candidate object of the pluralityof objects; determining, for the grip area of the gripper, a target zoneconfiguration of one or more zones of the plurality of zones thatcorresponds to an area of a top surface of the target candidate object;determining a grasp pose for the target candidate object, the grasp posealigning the target zone configuration of the gripper with the at leastone planar surface of the target candidate object; and executing thegrasp pose to lift the target candidate object by the gripper, thegripper activating each zone of the target zone configuration.
 2. Themethod of claim 1, wherein the plurality of objects comprise rectilinearobjects.
 3. The method of claim 1, wherein the plurality of objects arelocated in a walled container.
 4. The method of claim 3, wherein thegrasp pose avoids one or more walls of the walled container.
 5. Themethod of claim 4, wherein determining the grasp pose for the targetcandidate object comprises offsetting the grasp pose based on a locationof the one or more walls of the walled container.
 6. The method of claim3, wherein the operations further comprise: identifying a feature of thewalled container to avoid during removal of the target candidate objectfrom the walled container; and determining a motion path for removal ofthe target candidate object from the walled container, the motion pathavoiding the identified feature of the walled container.
 7. The methodof claim 1, wherein the operations further comprise determining that thegrasp pose includes minimum coverage of the target candidate object, theminimum coverage corresponding to an area providing suction forcesufficient to lift the target candidate object.
 8. The method of claim1, wherein the operations further comprise: determining that the gripperhas lifted the target candidate object a threshold distance; andresponsive to the determination that the gripper has lifted the targetcandidate object the threshold distance, activating all zones of thegripper.
 9. The method of claim 1, wherein the plurality of zonescascade across the plurality of vacuum suction cups of the gripper. 10.The method of claim 1, wherein the grasp pose corresponds to a positionof the gripper defined by a set of vacuum suction cups of the pluralityof vacuum suction cups overlapping the at least one planar surface ofthe target candidate object.
 11. A robot comprising: a visual sensor; agripper comprising a plurality of vacuum suction cups; and a controlsystem configured to perform operations comprising: receiving a minimumobject size for a plurality of objects varying in size, each object ofthe plurality of objects having at least one planar surface; dividing agrip area of the gripper into a plurality of zones based on the minimumobject size; locating, using sensor data captured by the visual sensor,a target candidate object of the plurality of objects; determining, forthe grip area of the gripper, a target zone configuration of one or morezones of the plurality of zones that corresponds to an area of a topsurface of the target candidate object; determining a grasp pose for thetarget candidate object, the grasp pose aligning the target zoneconfiguration of the gripper with the at least one planar surface of thetarget candidate object; and executing the grasp pose to lift the targetcandidate object by the gripper, the gripper activating each zone of thetarget zone configuration.
 12. The robot of claim 11, wherein theplurality of objects comprise rectilinear objects.
 13. The robot ofclaim 11, wherein the plurality of objects are located in a walledcontainer.
 14. The robot of claim 13, wherein the grasp pose avoids oneor more walls of the walled container.
 15. The robot of claim 14,wherein determining the grasp pose for the target candidate objectcomprises offsetting the grasp pose based on a location of the one ormore walls of the walled container.
 16. The robot of claim 13, whereinthe operations further comprise: identifying a feature of the walledcontainer to avoid during removal of the target candidate object fromthe walled container; and determining a motion path for removal of thetarget candidate object from the walled container, the motion pathavoiding the identified feature of the walled container.
 17. The robotof claim 11, wherein the operations further comprise determining thatthe grasp pose includes minimum coverage of the target candidate object,the minimum coverage corresponding to an area providing suction forcesufficient to lift the target candidate object.
 18. The robot of claim11, wherein the operations further comprise: determining that thegripper has lifted the target candidate object a threshold distance; andresponsive to the determination that the gripper has lifted the targetcandidate object the threshold distance, activating all zones of thegripper.
 19. The robot of claim 11, wherein the plurality of zonescascade across the plurality of vacuum suction cups of the gripper. 20.The robot of claim 11, wherein the grasp pose corresponds to a positionof the gripper defined by a set of vacuum suction cups of the pluralityof vacuum suction cups overlapping the at least one planar surface ofthe target candidate object.